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MULTI-VALUED  NEURAL  NETWORK  MODIFIED  MODEL  AND  / 

ITS  OPTICAL  REALIZATION1 

Zhu,  Weili2,  Chen,  Yansong3 

Abstract 

This  paper  presents  a  modified  optical  neural  network 
model,  and  the  optical  system,  constructed  with  spatial 
light  modulator  PROM,  can  materialize  the  associative  memory 
computation  of  such  kinds  of  models.  From  the  computer 
simulation  and  experimental  results,  it  is  clarified  that 
the  modified  model  improves  cognitive  ability  of  an  optical 
neural  network  and  also  improves  the  storage  capacity  to  a 
certain  extent. 

1 .  Introduction 

Since  at  the  early  part  of  the  '80s  .of  this  century 
Hopfield  proposed  his  neural  network  physical  model  [1]/ 
research  in  such  kind  of  computing  models  has  become  ever 
more  widely  spread  and  more  in  depth;  it  can  be  used  to 
solve  computation  problems  of  immense  amount  of  information 
[2],  and  also  can  be  used  to  simulate  the  associative  memory 
capability  of  the  human  neural  network,  and  as  long  as  the 
memory  bank  stays  below  15  %  of  the  total  number  of  neurons 
[1],  the  results  of  searched  memories  have  shown  sufficient 
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accuracy.  In  1985  Farhat  and  Psaltis  first  succeeded  in 
applying  optical  methods  to  the  realization  of  computation 
capability  of  the  Hopfield  neural  network  model  [3],  and 
initiated  a  new  branch  --  the  optical  neural  network  --  in 
the  neural  network  computation  science. 

Because  at  present  there  is  a  limit  in  the  direction  of 
optical  basic  equipment  capability,  it  is  still  rather 
difficult  to  realize  multi-value  or  negative  value  of  the 
Hopfield  model.  For  this  reason  one  cannot  help  to  bring 
the  mutually  connected  matrices  of  the  model  all  into  one 
(below  it  will  be  called  the  "return-to-one"  model)  in  order 
to  apply  the  binary  characteristics  of  instruments,  such  as 
membrane,  acoustic-optic,  or  electric-optic  modulator, 
liquid  crystal  spatial  light  modulator,  etc.  and  moreover  by 
increasing  the  number  of  devices  or  sizes  as  a  substitution 
for  solving  the  negative  value  problems  [ 3 ] .  Some  people 
use  the  internal  storage  computation  method  to  replace  the 
external  storage  computation  method  in  order  to  avoid  the 
appearance  of  negative  values  [4] .  The  above  kind  of 
modified  model  is  difficult  not  to  affect  cognitive 
capability  or  convergence  of  the  network. 

This  paper  proposes  a  modified  optical  neural  network 
model:  the  "Zheng-pu-tai"  model,  which  has  an  advantage  in 
improving  the  cognitive  ability  of  the  network  and  whose 
effect  is  clearly  visible  when  the  storage  capacity  is 
increased.  This  paper  first  explains  the  theory  of  this 
"Zheng-pu-tai"  model,  then  continues  on  to  carry  out  an 
analytic  evaluation  of  the  cognitive  capability,  followed  by 
an  introduction  of  the  method  to  optically  realize  such 
model,  and  finally  the  paper  presents  the  experimental 
results  in  order  to  carry  out  their  analyses. 
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2.  Presenting  the  "Zheng-pu-tai"  model  theory  and 
evaluating  its  cognitive  capability 

Suppose  there  are  M  monopole  2-dimensional  vectors  Vm 
of  N-unit  length  (m=l/2/ . . . ,M) ,  and  let  them  be  stored  in  a 
network  of  the  external  storage  method,  to  construct  the 
mutually  connected  matrices  of  the  "Zheng-pu-tai"  model  -as 
follows: 


27,  (i,  j)  -  Jj  [F“F"+  (1-PT)  (1-F7)] ,  d,  i- 1»  2,  •••,  N)  ( 1  > 

where  the  first  term  inside  the  square  parenthesis  is  the 
external  storage  matrices  of  the  stored  vectors,  while  the 
2nd  term  is  the  external  storage  matrices  of  their 
complimentary  vectors.  Obviously  all  the  elements  of  matrix 
Tc  are  non-negative  and  multi-valued.  By  use  of  one 
N-dimensional  2-component  vectors  V*  as  an  input  to  the 
network  to  carry  out  the  searching,  one  gets  the  output 
estimated  value  as  follows: 

j)m  *  (2) 

where  [  ]th  stands  for  the  threshold  of  neurons.  After  a 
repeated  substitution,  the  output  will  stabilize  onto  vector 
Vmo.  Generally  speaking,  Vmo  must  be  the  smallest  Hamming 
distance  stored  vector  of  the  input  vector  V*;  that  is,  it 
is  most  similar  to  V*. 

As  for  the  nature  of  the  monopole  2-component  vector 
adopted  by  this  paper,  one  may  refer  to  the  analytic  methods 
[5],  suggested  by  Farhat  and  Psaltis,  which  can  work  out  an 
evaluation  on  the  cognitive  capability  of  the  "Zheng-pu-tai" 
model  as  follows: 
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For  the  sake  of  mathematical  derivation  simplicity ,  it 
is  not  forbidden:  V*  =  Vmo  so  that  by  substituting  Eqn.  (1) 
into  Eqn. (2),  through  derivation  one  can  get 

fr-$  rr+  2  <*.Ff +&«•  ( 3 ) 

A  m+mo 


and  thus  the  output  signal-to-noise  ratio  becomes  as 


follows : 


8- 


N 


2  S  «-Fr 


(4) 


ctmonQ  which  ^  ^ 2  y  ^ 

*  *  2 0 

For  a  comparison,  the  results  obtained  with  the  Hopfield 
model  under  similar  conditions  are  as  follows: 

fr-csni,  i)Fr3«.o  <5> 

w 

T(i,j)-jz&VT-V  (2F“— 1)  -m»,  ( 6 } 

In  these  equations  the  terms  of  <5^  function  can  be 
satisfied  by  the  condition  of  Ti;j  =  0,  while  the  definition 
says  that  6^  =  1  (for  i  =  j )  or  0  (for  i  *  j).  With  the 
precondition  V*  =  Vmo  and  the  substitution  of  Eqn.  (6)  into 
Eqn. (5),  the  derivation  gets: 

tr*=  (N-M)rr+2  2  amVT+Cto  ( 7  ) 

m+mo 

One-time  reiterative  substitution  leads  to  the  output 
signal-to-noise  ratio  as  follows: 

g-  •?— - VTo  (8) 

2  S  <*mV? 

m+mo 

By  comparing  Eqn.  (4)  with  Eqn. (8),  one  can  see  that 
the  convergence  speed  of  the  " Zheng-pu-tai  model  is  faster 
than  that  of  the  Hopfield  model,  and  it  receives  very  little 
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direct  effect  from  M.  This  means  that  in  the  condition  to 
satisfy  M  <  (N/4)ln  N  [5],  these  two  differ  only  a  little. 
However,  when  M  increases,  the  signal-to-noise  ratio  of  the 
Hopfield  model  declines  drastically,  to  create  the 
requirement  of  more  iterative  substitutions  for 
computations,  and  also  due  to  the  decline  in  the  cognitive 
capability,  thus  the  stored  vectors  cannot  all  be 
recognized.  Consequently,  from  this  fact  one  can  see  that ^ 
the  allowable  stored  quantity  of  the  "Zheng-pu-tai"  model  is 
relatively  a  little  higher  and  this  point  will  be 
substantiated  in  the  following  experiments.  b±  and  cL  in 
Eqn.(3)  and  Eqn.(7)  are  all  constants,  and  their  values  are 
related  to  M  and  N,  and  thus  they  affect  the  selection  of 
the  threshold  but  do  not  affect  the  signal-to-noise  ratio. 


3.  Optical  realization  of  the  "Zheng-pu-tai"  model 

By  use  of  the  optical  system  shown  in  Fig.  1,  the 
optical  neural  network  "Zheng-pu-tai"  model  was  worked  out 
for  simulation  computations,  and  its  experimental  setup  is 
shown  in  the  photo  of  Fig.  2;  in  Fig.  1,  G  and  R  are  diode 
arrays  to  emit,  respectively,  green  and  red  colors,  and  the 
former  is  a  2-dimensional  array  to  be  used  as  the  external 
storage  matrix  of  the  input  stored  vector  for  the  system; 
and  "Zheng-pu-tai"  mutually  connected  matrices  Tci  were 
constructed  step— by— step  on  a  spatial  light  modulator  PROM; 
the  latter  is  a  1-dimensional  array,  to  be  used  as  the 
optical  output  of  vector  V*,  to  search  out  the  associative 
memory  for  the  system;  P  and  A  are,  respectively,  the 
polarizer  and  analyzer;  after  the  searched  results  have  been 
received  by  utilizing  CCD,  the  computer  system  helps  realize 
the  threshold  computation.  For  the  sake  of  simplicity,  the 
optical  photographic  elements  were  omitted  in  the  figure. 
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Fig.  1  A  of  optical  system  for  Fig.  2  A  photo  of  arrangement  for 

implementing  neural  network  optical  neural  network 

B.  red  T/Hm  array,  G.  green  LED  array,  P.  polarizor, 

A.  analyzor,  BG;  beam  splitter,  V:  DO  electrical  power 

Because  the  PROM  instrument  has  a  rather  wide  linear 
exposure  region  [6],  thus  it  can  pass  repeatedly  the 
exposure  rays  to  implement  the  accumulated  computations  of 
the  optical  matrices.  Every  time  the  G-LED  array  is  used  to 
expose  the  light,  there  exists  an  external  storage  matrix  of 
some  stored  vector  (or  its  complimentary  vector),  and  after 
2M  times  of  exposures,  one  gets  their  matrice-sum;  that  is, 
the  "Zheng-pu-tai"  mutually  connected  matrices  Tco  shown  by 
Eqn.  (1)  use  input  vector  V*  of  the  R-LED  array  to  search 
out  associative  memories  for  the  system,  to  realize  the 
multiplication  calculation  of  matrix  Tc  and  vector  V*  on 
PROM.  With  the  optical  information  received  by  CCD,  the 
input  computer  system  carries  out  the  threshold  computation, 
then  feeds  back  to  the  input  terminal  to  carry  out  the  next 
round  of  operations . 

Because  the  external  storage  matrix  of  each  stored 
vector  (or  the  complimentary  vector)  are  all  monopole 
2-dimensional  matrices,  thus  by  use  of  the  switching 
on-and-off  state  of  the  G-LED  one  can  conveniently  express 
two  value  of  l-and-0  of  the  matrix  elements,  and  the 
multi-valued  characteristic  of  the  mutually  connected  matrix 
Tc  is  to  rely  on  the  optical  accumulative  characteristics  of 
the  PROM  instrument.  Furthermore,  the  implementation  of  the 


6 


PROM  instrument  becomes  an  addition  to  the  specified 
computational  structure  of  the  " Zheng-pu-tai"  model  to 
promote  the  vitality  of  the  system,  which  is  useful  in 
constructing  a  dynamic  optical  neural  network. 


4.  Experimental  results  and  analyses 

By  use  of  the  "Zheng-pu-tai"  model  in  the  neural 
network  of  N  =  35  to  carry  out  a  computation  simulation,  the 
stored  vector  starts  from  M=6  (M/N  =  0.17)  to  increase  to 
M=12  (M/N  =  0.34).  According  to  the  description  given  in 
the  previous  section,  one  can  use  the  stored  vector  to  carry 
out  the  searching  job  (V*  =  Vmo)  for  the  network.  The 
computed  results  show  that  when  one  uses  a  "Zheng-pu-tai" 
model  PCM  to  compute,  the  stored  vectors  can  be  recognized 
in  100  times  out  of  100.  However  when  the  Hopfield  model  is 
being  used,  such  rate  drops  down  from  100,%  to  75%,  and  also 
the  time  of  repetitions  was  obviously  increased,  in 
accordance  with  the  previous  theoretical  estimations .  When 
the  return— to— one  model  is  used  in  the  computing,  this  rate 
also  drops  from  100  %  for  M  =  6  to  33%  for  M  =  12. 

By  use  of  the  optical  system  shown  in  Fig.  2  to  carry 
out  an  optical  simulation  for  the  "Zheng-pu-tai"  model,  the 
magnitude  of  the  network  is  N  =  8,  the  number  of  stored 
vectors  starts  out  at  M  =  2  to  increase  to  4,  and  V*  (*  Vmo) 
is  used  to  carry  out  the  searching.  As  shown  in  Table  1,  in 
the  optical  simulation  results  for  M  =  4,  "0"  means  an 
accurate  recognition  while  F  stands  for  searching  failure. 
For  the  sake  of  comparison,  the  computer  simulation  results 
carried  out  with  the  Hopfield  model  and  the  return-to-one 
model  under  similar  conditions  were  also  listed. 


Mil  Ita ■•wUw ">»lM *"*■"*  *[ TP”'“ 
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What  is  worth  mentioning  here  is  that  this  paper  is 
concerned  with  computations  by  adopting  stored  vectors  of 
fairly  good  orthogonality  and  ideal  input  vectors  (that  is 
to  say,  they  are  not  different  from  the  stored  vectors  or  at 
least  not  too  different).  What  has  already  been  discovered 
in  the  simulation  computations  is  that  when  the  stored 
vectors  of  a  little  bit  poorer  orthogonality  and  thfe  input 
vectors  of  rather  deformed  kinds  are  being  used,  the 
cognitive  ability  of  the  network  declines  accordingly. 
Furthermore,  when  the  "Zheng-pu-tai"  model  is  being  used, 
the  required  threshold  level  for  computing  Eqn.(2)  is  much 
higher  than  those  for  other  models. 
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